fix(pages-router): stream piped API responses with backpressure - #2735
fix(pages-router): stream piped API responses with backpressure#2735NathanDrake2406 wants to merge 5 commits into
Conversation
A handler that pipes into `res` (proxying an upstream, echoing a request body) could queue the entire stream in memory. The response bridge acknowledged every write immediately, so `write()` never returned false and a piped source was never paused, regardless of how slowly the client consumed the body. The Node production server also read API responses with `arrayBuffer()`, which held the full body in memory and deferred delivery until the source closed, so long-lived streams (e.g. proxied SSE) never flushed. Hold the write callback while the response body's queue is full and release it from the stream's pull hook. A held callback makes `write()` return false, which pauses any piped source until the consumer reads. Mark responses that are still being written when the handler settles and send them through the streaming path in the Node production server. Complete bodies (`res.json`, `res.send`, `res.end(data)`) keep the buffered path and its Content-Length behaviour.
commit: |
Performance benchmarksCompared 0 improved · 0 regressed · 6 within ±1.5%
View detailed results and traces 🟢 improvement · 🔴 regression · ⚫ change below 1.5% · paired base/head |
|
/bigbonk review for issues |
There was a problem hiding this comment.
Reviewed the backpressure mechanism, the stream-vs-complete decision, the prod-server branch, and marker propagation. The core design is sound and well-tested: I verified the _write parking + pull() release cycle works for serialized writes, that buffered responses (res.json, res.send, large res.end(data)) still complete because the reading adapter pulls and releases the parked callback, that no unhandled rejection leaks when a streaming handler rejects without a ctx, and that dev/prod parity now holds (the dev middleware already streamed via Readable.fromWeb().pipe(res); prod's arrayBuffer() buffering was the outlier this fixes). Cloudflare Workers streams API bodies natively so it inherits backpressure without needing the marker. Nice work.
One correctness-of-documentation issue and two smaller notes below. Nothing blocking.
Verification performed:
- Simulated the
desiredSize <= 0park /pull()release cycle — releases correctly on read, anddesiredSize ?? 1avoids deadlock on closed/errored controllers. - Confirmed
res.json/ largeres.end(data)/write()+end()are not marked streamed and complete fully. - Confirmed the reject-after-write and cancel-parked-write paths settle the parked callback with the error.
- Ran
tests/pages-api-route.test.ts,tests/pages-bodyparser-config.test.ts,tests/after-deploy.test.ts,tests/pages-request-pipeline.test.ts,tests/unified-request-context.test.ts,tests/pages-node-compat.test.ts— all green.
| if ( | ||
| this.controller && | ||
| !this.streamEnded && | ||
| !this.writableEnded && |
There was a problem hiding this comment.
The comment claims end(data) writes are finished immediately, but the !this.writableEnded guard doesn't actually achieve that. When res.end(data) runs, Node calls _write synchronously before end() sets writableEnded, so inside this _write this.writableEnded === false. I confirmed this empirically:
_write writableEnded=false <- the end(data) write
_final
after end() writableEnded=true
And end-to-end: a large res.end(bigData) parks its write callback and the end() completion callback does not fire until the body is read (endCallbackFired before read: false). So finish is gated on the first read for a large end(data), exactly what the comment says it avoids.
This is functionally harmless today — the buffering adapter reads the body via arrayBuffer(), which pulls and releases the parked callback, and the response is correctly not marked streamed (because finalizeResponse checks writableEnded, which is true after end() returns). But the stated rationale is inaccurate and could mislead a future maintainer who relies on end(data) writes not parking.
If you actually want end(data) writes to bypass parking, gate on writableEnding (set synchronously at the start of end(), unlike writableEnded) instead of writableEnded:
| !this.writableEnded && | |
| if ( | |
| this.controller && | |
| !this.streamEnded && | |
| !this.writableEnding && | |
| (this.controller.desiredSize ?? 1) <= 0 | |
| ) { |
Otherwise, please reword the comment to drop the "finish them immediately" claim, since parking-then-releasing-on-read is the real behavior.
| ) { | ||
| response.headers.set("content-type", result.defaultContentType); | ||
| } | ||
| await sendWebResponse(response, req, res, compress); |
There was a problem hiding this comment.
Minor behavioral note (not blocking): routing streamed API responses through sendWebResponse means compressible content-types now compress in "streaming" flush mode regardless of the 1024-byte COMPRESS_THRESHOLD that the buffered sendCompressed path applies. For a small streamed application/json/text/plain body this adds gzip/br overhead where the buffered path would have skipped it. SSE (text/event-stream) is unaffected since it's not in COMPRESSIBLE_TYPES. Probably fine — a live stream has no known length anyway — but worth a one-line comment so the divergence from the buffered threshold is intentional and documented.
| if (merged !== response) { | ||
| (merged as { __vinextStreamedApiResponse?: boolean }).__vinextStreamedApiResponse = ( | ||
| response as { __vinextStreamedApiResponse?: boolean } | ||
| ).__vinextStreamedApiResponse; |
There was a problem hiding this comment.
For consistency: the Cloudflare mergeHeaders in server/worker-utils.ts does not copy __vinextStreamedApiResponse (it only strips content-length for the HTML marker). This is not a bug — the Workers path returns new Response(response.body, ...) and streams natively, so it doesn't consume the API marker — but the asymmetry (prod-server + this pipeline propagate it, worker-utils drops it) is a latent trap if a future change makes the Workers adapter branch on the marker. Consider a brief comment in worker-utils.mergeHeaders noting the API marker is intentionally not propagated because Workers streams unconditionally.
|
Review posted successfully. Here's a summary of my review of PR #2735. Review SummaryThe PR fixes a real availability problem: piped Pages API responses ( What I verified (all green):
Findings (none blocking):
|
|
/bigbonk review for issues |
There was a problem hiding this comment.
Reviewed the backpressure mechanism, the stream-vs-complete decision in handlePagesApiRoute, the prod-server stream branch, and marker propagation. The core design is sound and the change correctly maps Node socket backpressure onto the Fetch ReadableStream boundary. I verified:
- Parking + release cycle:
_writeparks the callback only oncethis.controlleris set (which happens synchronously viaresolveOnce()on the first write) anddesiredSize <= 0. Since Writable serializes_write, there is at most onependingWriteat a time, so releasing one perpull()(and on_destroy) is correct and cannot leak.desiredSize ?? 1avoids deadlock on a closed/errored controller. - Buffered bodies still complete:
res.json/res.send/ largeres.end(data)are not marked streamed (checked afterwritableEndedis true) and the buffered adapter'sarrayBuffer()read pulls and releases any parked terminal write. - Error paths: handler-throws-after-write errors the stream via
res.destroyand reports once throughhandlerLifecycle; handler-throws-before-write still yields a 500 via the outer catch, and the losing race branch is handled (no unhandled rejection). - Workers lifecycle: the streaming branch returns before the handler settles, so registering
handlerLifecycleon the active execution context is the right fix; the newafter-deploy.test.tscovers the ambient/hybrid case. - Ran
tests/pages-api-route.test.ts,tests/after-deploy.test.ts,tests/pages-node-compat.test.ts,tests/pages-request-pipeline.test.ts— all green.
Contrary to the earlier bot review, the pages-node-compat.ts comment is accurate: it explicitly states end(data) writes can park, which matches the code. Two small non-blocking notes below.
| if ( | ||
| this.controller && | ||
| !this.streamEnded && | ||
| !this.writableEnded && |
There was a problem hiding this comment.
Edge case worth a note: a parked write is only released by a consumer pull() or by _destroy. If the returned Response body is never read and never cancelled, the final parked write stays parked, _final never runs, and any handler awaiting pipeline()/drain never settles (its promise is only kept alive via waitUntil). This mirrors real Node (an unconsumed socket also stalls the producer), so it's acceptable — but since this is the one path that can leave a handler promise pending indefinitely, a one-line comment here documenting that release depends on consumption would help future maintainers reason about it.
| ) { | ||
| response.headers.set("content-type", result.defaultContentType); | ||
| } | ||
| await sendWebResponse(response, req, res, compress); |
There was a problem hiding this comment.
Minor behavioral divergence (not blocking): routing streamed API responses through sendWebResponse means a compressible content-type (application/json, text/plain) now compresses regardless of the buffered path's 1024-byte COMPRESS_THRESHOLD. A small streamed JSON body will gzip/br where the buffered path would have skipped it. text/event-stream is unaffected (not in COMPRESSIBLE_TYPES). A live stream has no known length so this is defensible, but a one-line comment noting the threshold intentionally doesn't apply to streamed bodies would make the divergence explicit.
| if (merged !== response) { | ||
| (merged as { __vinextStreamedApiResponse?: boolean }).__vinextStreamedApiResponse = ( | ||
| response as { __vinextStreamedApiResponse?: boolean } | ||
| ).__vinextStreamedApiResponse; |
There was a problem hiding this comment.
Maintainability: __vinextStreamedApiResponse is now a magic string duplicated across pages-api-route.ts, pages-node-compat.ts (typed helpers), this file (inline cast), unified-request-context.ts (inline cast), and prod-server.ts (inline reader). pages-node-compat.ts already exports markVinextStreamedApiResponse / isVinextStreamedApiResponse; consider reusing those helpers (or a shared const for the key) at these copy sites so a future rename can't silently miss one. Mirrors the existing __vinextStreamedHtmlResponse pattern, so this is a pre-existing style choice rather than a new problem.
|
Review posted successfully. SummaryI reviewed PR #2735 (streaming piped Pages API responses with backpressure) and posted a batch review with 3 non-blocking inline comments. Verdict: The core design is sound and correct. The change properly maps Node socket backpressure onto the Fetch What I verified (all green):
Non-blocking findings:
I also noted that, contrary to the earlier bot review, the |
Overview
PagesResponseStreamowns backpressure;handlePagesApiRouteowns the stream-vs-complete decision; adapters only consume itres.json/res.send/res.end(data)behaviour is unchangedWhy
Node streams propagate backpressure by withholding the
_writecallback: a held callback makeswrite()returnfalse, which pauses anypipe()source.PagesResponseStream._writeenqueued each chunk into the WebReadableStreamand invoked the callback synchronously, so the Writable looked infinitely fast. A fast source piped intores(the proxy pattern #1902 enabled) kept producing at full rate while a slow client consumed nothing, and every unread chunk accumulated in the stream's unbounded queue. On real Next.js this cannot happen becauseresis a genuineServerResponsewith socket backpressure; the bridge silently dropped that property.Independently, the Node production server read API responses with
arrayBuffer()before sending. That predates streaming API bodies and was harmless while every API response was complete at resolution time. Once a response can resolve mid-stream, buffering holds the whole body in memory and defers the first byte until the source closes, so a proxied SSE stream never flushes.PagesResponseStreamdesiredSize <= 0and released from the stream'spull()hook (or on destroy)handlePagesApiRoutesendWebResponsestreaming path, with the sameapplication/octet-streamcontent-type fallback the buffered path appliedWhat changed
upstream.pipe(res)with a slow clientarrayBuffer(); first byte only after the source closedsendWebResponse; bytes flush as they are writtenres.json/res.send/res.end(data)mergeHeadersrewrap (same propagation as the streamed-HTML marker)Maintainer review path
packages/vinext/src/server/pages-node-compat.tsfor the backpressure mechanism (_writeparking,pull()release,_destroyrelease) and the marker helpers.packages/vinext/src/server/pages-api-route.tsfor the stream-vs-complete decision after the response settles.packages/vinext/src/server/prod-server.tsfor the stream-vs-buffer branch and content-type fallback.packages/vinext/src/server/pages-request-pipeline.tsandpackages/vinext/src/shims/unified-request-context.tsfor marker propagation across Response rewraps.tests/pages-api-route.test.tsfor the regression proof.Validation
res.jsonis not.tests/pages-api-route.test.tsandtests/pages-bodyparser-config.test.tspass, including the existing fix(pages-router): support stream proxying in API routes #1902 streaming, cancellation, and destroy-mid-stream cases.tests/pages-node-compat.test.ts,tests/pages-request-pipeline.test.ts,tests/unified-request-context.test.tspass (118 tests).vp checkclean on all touched files; pre-commit full check, staged tests, and knip passed.Risk / compatibility
res.write()implies chunked transfer.References